1. Technical Field
The present invention relates generally to an apparatus and method for reconstructing three dimensional (3D) faces based on multiple cameras and, more particularly, to an apparatus and method that are capable of effectively reconstructing 3D face images using color and depth images.
2. Description of the Related Art
Conventional 3D face reconstruction systems are chiefly intended for movies. Since the conventional 3D face reconstruction systems aim to reconstruct objects that are most similar to actual objects, almost all equipment is expensive, the installation of the systems is complicated, and the installation and use of the systems require experts. Conventional face reconstruction systems for medical applications are each equipped with multiple DSLR-class cameras and an expensive structured-light projector.
Furthermore, cameras and projectors used in such systems are of synchronous type, so that problems arise in that an additional device is required to achieve synchronization and in that reconstruction performance is rapidly deteriorated when synchronization is not established.
Meanwhile, recently, with the development of computer vision technology, there have been increasing attempts to reconstruct faces in three dimensions using inexpensive cameras such as mobile phone camera and web cameras. This technology has the advantage of being convenient, and may be used for applications for amusement because it is focused on entertainment. However, this technology is disadvantageous in that it cannot be used in the beauty and medical fields because of its low reconstruction quality.
A variety of experiments and research for acquiring the 3D depth information of an object using one or more depth cameras have been actively conducted. Since a low-price depth camera cannot receive synchronizing signals, a synchronization problem occurs when multiple heterogeneous depth cameras are used or one or more depth cameras are used along with another type of camera.
Conventional approaches for solving the problem of the synchronization between heterogeneous cameras include a scheme for performing synchronization using sound, and a scheme for indicating time codes in captured image frames and then performing synchronization. However, the scheme for performing synchronization using sound is disadvantageous in that it is difficult to automate the scheme and synchronization should be performed after the capturing of images by manually comparing sound with image signals through post processing. The scheme for performing synchronization using time codes is also performed by comparing time codes through post processing step and aligning images having the same time code. However, this scheme can be automated because comparing only time codes or time stamps is sufficient. Accordingly, in order to acquire images while automatically synchronizing the images during capturing and reconstruct the 3D appearance of an object using the acquired images, it is difficult to use the conventional schemes.
Meanwhile, in order to create 3D content for stereoscopic TV such as 3D TV or multi-view TV, a scheme was proposed for constructing a heterogeneous multi-camera system using a depth camera and a high-resolution camera together and improving a low-resolution depth map acquired by the depth camera using information acquired by the high-resolution camera.
In the above applications, synchronization is essential when heterogeneous cameras are used together, and a depth camera capable of supporting synchronization is very expensive. Accordingly, the system is very expensive, and the installation and environmental settings of the system require a long time and professional knowledge.
Korean Patent Application Publication No. 2007-0009899 discloses an apparatus for creating a 3D model on the assumption that synchronized images are received via multiple stereo cameras. The technology disclosed in the publication has some limitations in that it is impossible to use an asynchronous camera and received images should have the same resolution.
Accordingly, there is an urgent need for a new technology that can reconstruct 3D images.